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Abstract 

United  States  defense  operations  are  greatly  enhanced  by 
satellites,  however  these  key  assets  are  vulnerable  to  perils 
such  as  space  weather  or  acts  of  aggression.  Unfortunately 
it  is  not  only  difficult  to  defend  against  such  threats,  it  can 
be  difficult  to  determine  the  cause  from  the  ground.  What 
may  at  first  appear  to  be  a  routine  system  glitch  may  in  fact 
be  something  much  more  serious.  This  situation  is 
aggravated  by  the  fact  that  Air  Force  satellite  control  centers 
use  antiquated  technology  requiring  multiple  human 
controllers  per  satellite,  each  viewing  alphanumeric  displays 
that  degrade  situational  awareness,  increase  crew  workload 
and  invite  confusion  during  demanding  wartime  scenarios. 
The  Air  Force  Research  Laboratory  (AFRL/F1ECP)  in 
conjunction  with  various  organizations  at  Schriever  Air 
Force  Base  are  conducting  research  to  increase  situational 
awareness  of  the  orbital  battlespace  by  allowing  operators  to 
navigate  through  three-dimensional  displays  with  voice- 
activated  commands.  This  speech  interface  to  Satellite  Tool 
Kit  -  which  will  be  discussed  in  this  paper  -  is  intended  to 
be  an  initial  step  at  enabling  operators  to  quickly  gather 
information  from  satellites  or  constellations.  Future 
research  plans  call  for  applying  usability  engineering 
techniques  to  the  satellite  attack  identification  and  reporting 
process,  and  then  applying  the  optimal  configuration  of 
human  interface  technologies. 

Introduction 

Satellites  are  highly  autonomous  systems  that  usually 
require  relatively  little  control  from  the  ground.  When 
human  intervention  is  required,  controllers  rely  on 
information  transmitted  between  the  satellite  and  their 
control  station  via  telemetry  links.  Many  of  today’s  Air 
Force  controllers  are  still  viewing  telemetry  on  antiquated 
1960’s  vintage  workstations  with  alphanumeric  text 
displays  that  degrade  situational  awareness,  aggravate  crew 
workload  and  invite  confusion  during  on-demand  wartime 
scenarios.  Often  multiple  controllers  are  employed  to 
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ensure  erroneous  inputs  are  not  made.  Figure  1  shows  a 
typical  screen  used  by  satellite  controllers  that  are  often 
cluttered  with  cryptic  mnemonics  and  inconsistent  data 
values.  A  new  generation  of  human  interface  technologies 
is  sorely  needed  and  indeed  some  efforts  are  being  made  to 
improve  the  situation.  However  few  of  these  efforts  use 
proven  human  factors  engineering  principles. 


Figure  1:  Example  of  a  satellite  operator  screen  employing 
ineffective  use  of  color  and  inexplicit  mnemonics 


Improved  human-computer  interface  technologies  will  be 
particularly  needed  during  critical  mission  events  such  as 
when  satellites  are  attacked.  For  this  reason  and  others,  Air 
Force  Space  Command  (AFSPC)  is  undergoing  a  paradigm 
shift  from  space  surveillance  to  space  situational 
awareness.  (AFSPC,  2002)  In  other  words,  there  is  a  need 
to  fully  understand  in  near  real-time  what  is  happening 
with  all  space  assets  as  opposed  to  passively  monitoring 
individual  satellites.  The  ultimate  goal  is  to  allow 
warfighters  to  have  a  clear  understanding  of  the  space 
landscape  to  confidently  discriminate  between  intentional 
and  unintentional  effects  on  space  systems  and  the 
capabilities  they  provide.  (General  Dynamics,  2001)  The 
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warfighter  must  be  able  to  perform  “intel  preparation  of  the 
battlefield”  by  characterizing  friendly  and  adversary  space 
system  status,  capability  and  actions.  When  the  availability 
of  one  or  more  satellites  is  compromised  -  whether 
innocent  or  intentional  circumstances  -  it  will  be  critical  to 
effectively  detect,  identify,  locate,  classify,  assess  and 
report  the  incident.  For  brevity’s  sake  I  will  refer  to  these 
actions  collectively  as  “processing”  the  incident.  This 
processing  needs  to  take  place  at  several  organizational 
levels  from  the  satellite  operator  up  to  the  commander  in 
chief  (figure  2). 


Past  Research 

With  any  form  of  applied  science,  it  is  important  to 
thoroughly  understand  the  area  in  which  technology  will  be 
applied  before  coming  up  with  a  technological  solution. 
Therefore  before  conducting  research  in  space  situational 
awareness,  the  Air  Force  Research  Laboratory,  Fluman 
Effectiveness  Division  (AFRL/F1E)  has  conducted  research 
in  routine  satellite  operations.  This  research  and  other 
relevant  research  are  summarized  below. 


RF/Laser 
Space  Weather  ~ 
Other  Threats 


Commander 
in  Chief 


Response 

Approval 


Figure  2:  Satellite  Threat  Warning  and  Attack  Reporting 
process  with  the  conceptual  integration  of  intelligent  agents 


This  concern  for  space  capability  protection  is  not 
unfounded.  The  events  of  September  11,  2001  showed  the 
American  people  that,  in  the  blink  of  an  eye,  our  enemies 
could  significantly  impact  our  national  security.  If  a 
coordinated  act  of  aggression  would  be  directed  at  our 
space  assets,  our  defense  capabilities  could  be  hindered  and 
it  would  likely  come  at  a  time  when  these  capabilities  are 
most  needed.  Granted,  a  space  attack  requires  much  more 
sophistication  than  hijacking  a  commercial  airliner,  but  the 
steaks  are  too  high  not  to  be  ready  for  such  an  event.  New 
technologies  in  space  vehicle  design,  sensors  and  materials 
are  clearly  important  for  future  space  protection  but  the 
ability  of  the  warfighter  to  understand  the  situation  could 
be  just  as  critical.  If  we  are  not  able  to  head  off  an  attack, 
we  must  at  least  be  able  to  know  how  much  our  capability 
has  been  degraded,  what  advantage  is  trying  to  be  gained, 
and  who  or  what  is  responsible.  Taking  days  to  process  the 
information  will  likely  cause  us  to  lose  critical  time  to  react 
to  an  incident. 


Research  in  Satellite  Operations 

Although  space  capability  protection  is  a  relatively  new 
thrust,  research  in  crew  interfaces  for  satellite  control  is  not 
new.  From  April  2000  to  August  2002,  AFRL/FIECP 
(Fluman  Interface  Technology  Branch)  worked  with 
Monterey  Technologies,  Inc.  (MTI)  on  a  Small-Business 
Innovative  Research  (SBIR)  program  to  develop  innovative 
interface  concepts  for  satellite  controllers.  In  this  task  MTI 
videotaped  a  task  analysis  of  a  typical,  unclassified  satellite 
“pass”  (i.e.,  the  time  when  a  controller  has  a  real-time 
telemetry  link)  then  developed  new  interface  concepts  and 
implemented  a  testbed  in  which  to  test  these  concepts.  As 
shown  in  figure  3,  this  analysis  revealed  that  manual 
(mouse  clicks  and  keyboard  input)  and  visual  modalities 
were  heavily  loaded  whereas  the  speech  and  hearing 
modalities  were  light. 


Percentage  of  Tasks  Using  Each  Modality 
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Figure  3:  Satellite  Controller  Modality  Analysis 


Once  the  task  analysis  was  completed,  interface  concepts 
were  developed  on  the  testbed  that  consisted  of  better 
graphical  user  interfaces,  touch  screens,  speech  input  and 
synthetic  voice  feedback.  Once  completed,  the  new 
interfaces  were  tested  in  relation  to  the  baseline  system. 
The  results  showed  a  marked  improvement  in  operator 
performance  with  approximately  one  sixth  the  number  of 
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steps  required  and  less  than  half  of  the  time  to  perform  the 
tasks.  (Sharkey,  et  al  2002) 


Research  with  Speech  Interfaces 

Long  before  the  research  by  MTI,  the  Air  Force  has 
investigated  speech  interfaces  for  communication  between 
an  aircraft  pilot  and  the  “electronic  crewmember.” 
(Williamson,  1996)  These  studies  showed  that  speech  was 
a  viable  technologies  even  in  this  less  than  serene 
environment  of  a  fighter  cockpit  which  can  be  noisy  and 
put  stresses  on  the  pilot  that  are  typically  not  experienced 
by  satellite  controllers.  In  the  relatively  tranquil 
environment  that  a  satellite  controller  works  in,  speech 
input  has  demonstrated  considerable  potential  to  streamline 
interfaces,  reduce  task  times  and  cut  down  the  learning 
curve.  Although  the  research  at  AFRL/HE  is  just 
underway,  space  organizations  have  expressed 
considerable  interest  in  applying  speech  recognition  in  the 
near  term  for  satellite  control. 

Recent  advances  in  this  technology  have  allowed  speech 
recognition  without  customizing  or  training  the  software 
for  an  individual’s  voice.  Although  many  of  us  complain 
that  we  seldom  talk  to  real  support  people  anymore,  these 
advances  have  made  it  possible  for  commercial 
organizations  to  use  speech  recognition  on  a  massive  scale 
to  lessen  the  burden  on  live  telephone  operators.  We 
believe  it  is  only  a  matter  of  time  when  speech  input  will 
be  used  for  personal  computers  including  new  tablet 
computers. 


Autonomous  Systems 

Few  systems  today  are  fully  autonomous.  There  is 
typically  a  need  for  humans  to  interact  with  the  system  in 
certain  circumstances.  Table  1  from  Sheridan  (1992) 
breaks  automation  out  into  ten  discrete  levels.  Note  that  all 
but  level  1 0  involve  some  human  involvement.  The  further 
down  the  scale,  the  more  continual  attention  is  required 
from  the  human.  This  scale  can  be  useful  for  designers  of 
human  interfaces  for  autonomous  systems  of  all  types  to 
better  understand  how  to  approach  the  interface  design. 

Clearly  autonomy  can  allow  humans  to  accomplish  things 
never  before  possible  but  one  common  mistake  in 
developing  highly  autonomous  systems  is  to  overlook  the 
needs  of  the  human  when  things  go  wrong.  AFRL/F1E  has 
gained  a  much  better  understanding  of  this  problem  in  our 
work  with  unmanned  air  vehicles  (UAVs).  UAVs  -  many 
of  which  are  highly  autonomous  -  are  increasingly 
important  to  our  national  security.  With  their  automation, 
UAVs  are  very  much  like  satellites  -  especially  from  a 
controller’s  standpoint.  This  fact  may  make  it  possible  in 
the  future  for  one  person  to  control  vehicles  in  the  air  and 
in  space. 


Table  1:  Levels  of  Automation 


Level 

Action  performed  by  the  computer. 

The  computer... 

HIGH 

10 

Decides  everything  and  acts  without  human 
involvement 

9 

Informs  the  human  only  if  it,  the  computer, 
decides  to 

8 

Informs  the  human  only  if  asked  to 

7 

Executes  automatically  then  must  inform  the 
human 

6 

Allows  the  human  a  restricted  time  to  veto  before 
automatic  execution 

5 

Executes  the  suggestion  if  human  approves 

4 

Suggests  one  alternative 

3 

Narrows  selection  down  to  a  few 

2 

Offers  a  complete  set  of  alternatives 

LOW 

1 

Offers  no  assistance:  human  makes  all  decisions 
and  performs  all  actions 

In  designing  an  optimal  interface  to  support  UAV 
operations,  experiments  have  been  conducted  to  determine 
the  value  of  basic  two-dimensional  (2-D)  displays,  flat 
perspective  view  displays  and  multi-sensory,  3-D 
immersive  displays  promoting  "virtual  presence."  Just  as 
with  our  satellite  research,  our  UAV  research  initially 
focused  on  enhancing  existing  ground  stations  by 
improving  existing  interfaces.  Ultimately,  a  new 
perspective  view  and  immersive  interface  concepts 
supporting  future  unmanned  vehicles  may  be  developed. 
This  advanced  interface  is  expected  to  reduce  operator 
workload,  increase  operator  situational  awareness,  and 
improve  overall  system  performance  in  current  and  future 
UAVs.  (Draper  2002) 

Relevant  Activity  In 
Space  Situational  Awareness 

Research  at  CERES 

The  Center  for  Research  Support  (CERES)  at  Schriever  Air 
Force  Base,  Colorado  is  developing  concepts  for  future 
Satellite  Operations  Squadrons  (SOPS)  to  effectively  react 
to  satellite  attacks.  In  this  effort,  called  the  Defensive 
Counterspace  Testbed  (DTB),  CERES  is  aiming  to 
improve  the  process  of  satellite  threat  identification  and 


attack  reporting.  But  since  satellite  problems  will  rarely  be 
attributed  to  acts  of  aggression,  methods  will  be  needed  to 
clearly  distinguish  critical  events  from  normal  system 
problems  or  space  weather  incidents.  The  current  thinking 
is  to  use  data  fusion  and  artificial  intelligence  to  derive  a 
“best  guess”  as  to  the  cause  and  assign  a  confidence  or 
certainty  factor.  But  to  my  knowledge  there  have  been  no 
studies  as  to  how  this  information  should  be  displayed. 

Although  human  interface  design  is  not  the  focus  of  the 
DTB  effort,  CERES  is  creating  a  prototype  system  to 
investigate  some  possible  displays.  For  satellite  state-of- 
health  monitoring,  in-house  developed  screens  are  being 
used.  These  satellite  operations  screens  are  a  major 
improvement  over  the  screens  shown  in  figure  1  because 
they  are  much  more  intuitive  and  graphically  friendly.  To 
improve  the  controller’s  situational  awareness  of  the  space 
environment,  Satellite  Tool  Kit®  (STK)  by  Analytic 
Graphics,  Inc.  is  to  provide  two-dimensional  and/or  three- 
dimensional  (3-D)  displays  of  constellations  or  individual 
satellites.  MATLAB®  by  The  MathWorks  Inc.  is  currently 
being  proposed  to  display  space  weather.  CERES  believes 
it  will  be  best  to  allow  the  user  to  have  some  screens 
persistent  so  they  are  currently  proposing  to  use  a  wall  of 
displays. 

When  a  significant  event  occurs  that  causes  a  change  in  the 
satellite’s  health,  the  plan  is  for  an  STK  window  to 
automatically  pop-up  to  show  the  affected  satellite  in  3-D. 
In  addition  to  depicting  the  location  and  orientation  of  the 
satellite  with  respect  to  Earth,  it  is  also  possible  to  display 
information  such  as  uplink  and  downlink  status  and 
possibly  unfriendly  satellites  in  the  vicinity. 


Research  at  AFRL 

AFRL/HE  is  working  closely  with  CERES  to  provide 
expert  human  factors  support.  The  first  phase  of  HE’s 
research  will  draw  on  the  experience  gained  through 
previous  interactions  with  space  organizations  and  the  MTI 
SBIR.  Depending  on  the  type  of  attack,  the  Satellite 
Operations  Squadrons  are  likely  the  first  place  where  a 
satellite  attack  will  be  detected.  AFRL  researchers  are 
therefore  aiming  to  improve  the  accuracy  and  tempo  to 
process  a  satellite  attack. 

The  specific  goal  of  the  AFRL  research  this  year  will  be  to 
apply  human  interface  technologies  to  allow  a  controller  to 
manipulate  visual  display  using  spoken  commands.  This  is 
an  important  capability  to  allow  the  controller  to  see 
exactly  what  they  need  to  see  without  frustrating  keyboard 
commands  or  menu  options.  One  of  the  biggest  benefits  of 
speech  interfaces  is  the  ability  to  encapsulate  multiple 
commands  into  one  logical  phrase.  For  example,  if 
multiple  satellites  are  attacked,  the  controller  could  be 
inundated  with  windows  showing  each  satellite  under 
attack.  Rather  than  traverse  through  pull-down  menus  or 
sift  through  overlapping  windows,  the  speech  interface  will 


allow  the  user  to  ask  the  computer  to  display  a  certain 
satellite  or  to  tile  certain  windows.  As  shown  in  figure  4,  a 
controller  may  ask  to  see  satellites  that  have  certain 
suspicious  behaviors  such  as  low  power  or  telemetry 
transmission  problems. 


Specifically  these  are  the  first  questions  AFRL  is 
addressing  in  this  research: 

1.  How  can  voice  input  improve  the  operations?  The 


Figure  4:  Speech  Input  for  Satellite  Operations 


hypothesis  is  that  voice  input  will  eventually  be  more 
intuitive  and  will  free  up  the  hands  for  other 
interactions  such  as  manipulating  objects  or  views 
(cameras)  in  the  virtual  3-D  displays.  Voice  input  will 
be  used  for  commands  such  as  “show  a  close  up  of 
satellite  X”  or  “show  all  satellites  on  orbital  plane  Z”  or 
“tell  me  what  subsystems  have  been  affected  on  satellite 
Y.” 

2.  Should  the  STK  windows  of  individual  satellites  pop  up 
automatically  when  the  computer  detects  an  abnormal 
state  or  should  the  user  request  the  information  using 
keyboard/mouse  or  voice  commands? 

3.  How  should  these  windows  be  displayed  in  relation  to 
other  windows  that  CERES  has  determined  to  be 
important  for  the  controller?  These  other  windows 
include  STK  worldview,  MATLAB-based  space 
weather  display,  satellite  control  screens  developed  at 
CERES,  and  the  view  for  higher  headquarters  (on  up  to 
the  Commander  in  Chief). 

Future  Research.  In  future  phases  of  this  research,  we 
hope  to  apply  proven  usability  engineering  methodologies 
to  the  satellite  threat  warning  and  attack  reporting  process. 
Usability  engineering  is  a  structured  process  to  both 
develop  new  interfaces  and  improve  existing  interfaces. 
(Nielsen,  1994)  Many  major  companies  have  found  that  the 
cost  to  perform  usability  engineering  is  worth  the 
investment  for  their  Internet  screen  designs  and  other 


human  interfaces.  As  with  these  companies,  it  may  not  be 
practical  to  perform  all  of  the  steps  in  the  complete 
usability  engineering  process,  however  it  should  at  least  be 
possible  to  do  a  quick  task  and  functional  analyses  and  a 
couple  iterative  designs. 

We  also  hope  to: 

1 .  Investigate  the  utility  of  virtual  reality  displays  and 
interactivity.  The  CAVE  -  a  fully  immersive  virtual 
reality  room  (Figure  5)  -  is  one  possibility  for  this 
research.  The  CAVE  will  allow  an  operator  or 
commander  have  a  god’s  eye  view  of  all  space  and 
ground-based  assets.  In  addition,  body  tracking  and 
gesture  recognition  may  be  investigated. 


Figure  5:  The  CA  VE  provides  fully  immersive  virtual 
reality 


2.  Investigate  the  utility  of  head-mounted  displays  to 
increase  screen  “real  estate”  as  was  done  in 
AFRL/HECP’s  ongoing  Airborne  Warning  And 
Control  System  (AW ACS)  research. 

3.  Investigate  HCI  concepts  for  higher  headquarters 
including  North  American  Aerospace  Defense 
Command  (NORAD)  and  U.S.  Space  Command.  It 
is  believed  that  providing  3-D  views  would  be 
beneficial  to  these  organizations  but  they  will 
probably  require  the  ability  to  get  more  detailed 
information  on  demand. 

4.  Determine  how  the  High  Level  Integrated  Team 
Training  Environment  for  Space  (HILITE)  software 
can  be  used  for  Space  Situational  Awareness  and 
Defensive  Counterspace.  In  particular  HILITE  uses 
intelligent  agents  and  automated  message  routing 
that  could  prove  to  be  invaluable  for  SSA. 
Intelligent  agents  are  virtual  personal  assistants  who 
act  autonomously,  adaptively  and  cooperatively  to 
seek  information  over  a  network.  Intelligent  agents 
can  tirelessly  act  on  behalf  of  the  user  by 
continually  monitoring  situations,  performing 


certain  tasks,  and  adapting  to  new  work  patterns  and 
unanticipated  operational  events. 

5.  Apply  Work  Centered  Support  System  (WCSS) 
concepts.  AFRL/HEC  and  the  Sustainment 

Logistics  Branch  (AFRL/HESS)  are  jointly 
developing  this  technology,  which  also  applies 
intelligent  agents.  Intelligent  agents  are  applied 
using  work-centered  analysis  and  design  techniques 
after  conducting  process  task  and  cognitive  work 
analyses.  Process  task  analysis  involves  applying 
traditional  physical  and  information  processing 
analyses.  Cognitive  work  analysis  involves 

analyzing  mental  work,  problem  solving,  and 
dynamic  work  behavior  for  the  job.  Work-centered 
design  aims  to  support  work  in  context  through 
appropriate  display  and  aiding  techniques.  It 
provides  “decision  quality”  information  that  is 
context  tailored,  aids  the  user  in  performing  routine 
tasks,  and  rapidly  adapts  to  atypical  work  patterns. 
(Eggleston  and  Whitaker,  2002) 

Conclusions 

Current  crew  interfaces  for  Air  Force  satellite  operations 
are  considered  by  many  to  be  insufficient  for  routine  use  let 
alone  during  stressful  mission  critical  situations.  For 
critical  situations,  interfaces  are  needed  that  allow  users  to 
gain  situational  awareness  quickly  about  the  orbital 
battlespace  to  differentiate  between  routine  and  critical 
situations.  If  a  situation  is  deemed  critical,  the  warfighter 
needs  to  be  able  to  assess  the  impact  and  intent  so  that 
appropriate  actions  can  be  taken.  The  research  being 
performed  by  space  organizations  and  AFRL  are  important 
steps  in  making  this  happen. 
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